Putting small language models under the microscope
We’ve spent the past few years discussing large language models, the huge AI models that power a lot of the generative AI tools that have dominated the headlines.
But small language models are also possible – and rapidly growing in popularity.
What benefits do these tiny AI models offer, and how much use will they get in the coming months and years, and how do they differ from other lightweight, ‘low latency’ models?
In this episode, Jane and Rory take a look at some of the smallest AI models on the market, asking what they’re for and if they could be the future of the technology.
Highlights
“Microsoft gave the example of using them for organizations with very limited resources, using them for very specific needs at the edge. So for example, if you wanted to put small language models on smartphones or in a vehicle, they could say very quickly answer user prompts without needing to send any data to the cloud or even have to stop and process the requests.”
“Distillation, in short, is taking a small model, typically an open source, quite lightweight, small language model and training it based on outputs from a much larger model.”
“It feels very much like you’re pulling something apart only to put it back together again in a possibly more expensive or less convenient way. The reality is, if you need the capability of one of the hyperscalers, you probably don’t have the capability to do that yourself, or it wouldn’t make any economic sense to do so.”
Subscribe
TOPICS
Source link
